Scalable and accurate variational Bayes for high-dimensional binary regression models

Summary Modern methods for Bayesian regression beyond the Gaussian response setting are often computationally impractical or inaccurate in high dimensions. In fact, as discussed recent literature, bypassing such a trade-off is still an open problem even routine binary models, and there limited theory on quality of variational approximations high-dimensional settings. To address this gap, we study approximation accuracy routinely used mean-field Bayes solutions probit with priors, obtaining novel practically relevant results pathological behaviour strategies uncertainty quantification, point estimation prediction. Motivated by these results, further develop new partially factorized posterior distribution coefficients that leverages representation global local variables but, unlike classical assumptions, it avoids fully approximation, instead assumes factorization only variables. We prove resulting belongs to tractable class unified skew-normal densities crucially incorporates skewness and, state-of-the-art solutions, converges exact density $p \rightarrow \infty$. solve optimization problem, derive coordinate ascent inference algorithm easily scales $p$ tens thousands, provably requires number iterations converging $1$ Such findings also illustrated extensive empirical studies where our solution shown improve any $n$ $p$, magnitude gains being remarkable those $p>n$ settings impractical.